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DISPLAY DEVICE AND METHOD FOR DETERMINING AN AREA OF 



IMPORTANCE IN AN ORIGINAL IMAGE 

CROSS-REFERENCE TO RELATED APPLICATIONS 
5 Reference is made to commonly assigned, co-pending U.S. Serial 

No. 10/061,385, entitled SYSTEM AND METHOD OF PROCESSING A 
DIGITAL IMAGE FOR USER ASSESSMENT OF AN OUTPUT IMAGE 
PRODUCT, filed February 1, 2002 in the name of John R. Randall, et al.; U.S. 
Serial No. 10/745,006, entitled DISPLAY DEVICE WITH AUTOMATIC AREA 
1 0 OF IMPORTANCE DISPLAY, filed December 23, 2003 in the name of Susan J. 
Beck; and U.S. Serial No. 10/292,684, entitled CAMERA HAVING 
CONTINUOUSLY CROPPING VIEWFINDER, filed November 12, 2002. 

FIELD OF THE INVENTION 
The present invention relates to imaging methods and systems that 
1 5 determine an area of importance in an original image. 

BACKGROUND OF THE INVENTION 
Digital images are becoming an increasingly popular form of info- 
imaging. One reason for this popularity is the ease with which users can 
manipulate, edit, alter and enhance such digital images after capture. For 
20 example, users often use manual digital imaging and editing tools, such as the 

crop and zoom tools provided in the Kodak Picture CD software sold by Eastman 
Kodak Company, Rochester, N.Y., U.S.A., to improve the appearance of digital 
images. These image editing tools allow a user to limit the image content of an 
image to emphasize important elements in the image. Other image editing, sorting 
25 and other tools can also be usefully applied to portions of images that a user 
considers to be important after the image has been captured. However, 
photographers typically make area of importance designations using such tools 
only on selected images during the post capture phase. 

It is often useful to associate an area of interest determination with 
30 an image at the time that an image is captured, shared or stored. Building such an 
association at the time of image capture or during use allows the image to be 
processed, stored, sorted, and/or shared in a manner that is determined at least in 



part based upon with the area of importance information. Further, to the extent 
that that a such a designation can be made in a simple fashion at capture there is a 
greater likelihood that the photographer will make such a designation for more 
images and that such designations will be more accurate. In addition, with the 
5 increased use of digital cameras and other display devices that enable images to be 
captured or otherwise obtained, and immediately transmitted a remote recipient, 
what many users also desire a simple and effective means for manually 
designating what is important in an image that can be quickly and efficiently 
executed. 

1 0 Automatic and semi-automatic image processing and editing 

algorithms are known. These can be applied to enhance the appearance of a 
digital image while requiring little user input. These automatic and semi- 
automatic image processing algorithms analyze the content of an image and apply 
various assumptions about what a user would likely find to be important elements 

15 of an image. 

In a first example of an automatic algorithm, large oval shaped 
objects having color that approximates known flesh tones can be assumed to be 
important to the user. The degree of presumed importance can be increased 
where, for example, the large oval face shaped objects are positioned near the 

20 center of an image. See for example, commonly assigned U.S. Patent No. 

6,282,317, entitled "Method For Automatic Determination of Main Subjects in 
Photographic Images" filed by Luo et al. on December 31, 1998. Other 
algorithms use frequency analysis of the digital data that forms digital images to 
identify elements of an image that are considered to be of greater importance. 

25 Such algorithms make assumptions about what is important in an image based 
upon analysis of the visual elements of the captured image. See for example 
commonly assigned U.S. Patent Application Serial No. 09/176,805 entitled 
"Determining Portions of a Digital Image Which are In Focus" filed by Erkkilea 
et al. on October 22, 1998. 

30 In still another example of an automatic system is described in 

commonly assigned U.S. Patent Application Serial No. 10/303,978, entitled, 
"Digital Imaging System With Eye Monitoring" filed November 25, 2002, by 
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Miller et al. This application describes an image capture system having an eye 
monitoring system that stores eye information including eye gaze direction 
information during an image capture sequence and associates the eye information 
with an image captured during the image capture sequence. In certain 
5 embodiments, context information is also captured with the eye information. The 
context information is also associated with the image. The eye monitoring system 
described therein is useful for its intended purpose and has broad application. 
However, some consumers prefer not to use eye monitoring systems. 

In a further example of an automatic system is described in 

10 commonly assigned U.S. Patent Application Serial No. 10/324,489, entitled 
"Imaging Method and System For Determining An Area Importance In An 
Archival Image" filed by Fredlund on December 20, 2002.. In this application an 
imaging system is provided having an image source operable to obtain evaluation 
images and an original image. A controller causes the image source to obtain a set 

1 5 of evaluation images and an original image. A signal processor compares the 

evaluation images and the original image to identify portions of the original image 
that are common to all of the captured images. The signal processor stores area of 
importance data identifying the portions of the original image that correspond to 
the common portions and associates the area of importance data with the original 

20 image. 

Commonly assigned U.S. Patent No. 6,516,154 entitled "Image 
Revising Camera and Method" filed by Parulski et al. on July 17, 2001 also 
describes an automatic method wherein a camera stores an original image in a 
memory and analyzes image to determine a plurality of parameters. The 

25 parameters are matched to one or more of a plurality of editorial suggestions to 
define a suggestion set. One of the potential set of suggestions includes a change 
in the active area of the original image that is incorporated into a final print. In 
this camera, the change in active area is suggested to the camera operator who 
can, in turn select or reject the suggested change. 

30 While such automatic algorithms are often very useful, some users 

prefer to manually designate areas of importance in an image. Accordingly, a 
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variety of cameras provide compositional suggestions to users that intend to limit 
an image to a include important information. 

For example, U.S. Patent No. 5,706,049 entitled "Camera that 
records and Active Image Area Identifier With an Image' 1 filed by Moghadam et 
5 al. on November 30, 1995, describes a camera that includes an image receiver for 
capturing an image of an object, an optical section for directing image light from 
the object to the image receiver, and a viewfinder for viewing the object prior to 
capture together with a tile pattern for specifying active hotspot areas in the 
viewed image. The tile pattern in the viewfinder is composed of a plurality of 

10 individual tile areas that are visible through viewfinder together with the image of 
the object. The camera further includes means for designating one or more 
individual tile areas and active areas of the image, and means for recording a 
location all the active areas in a memory location that is accessible to act 
apparatus external to the camera. The '049 patent describes a digital camera 

1 5 embodiment having a thumbwheel switch which acts either as a tile pattern 

selection switch for selecting a particular pattern to be stored in a pattern memory 
or a tile area designation switch for selectively sequentially highlighting one or 
more of the stored tile area patterns, until a desired active area is highlighted. A 
hotspot selection switch is engaged to select one of the designated hotspot areas; 

20 The '049 patent, thus is used to designate an area of a captured original image that 
can be used to perform a function of a hotspot which can be integrated with 
integration of a display device, presentation system or an HTML browsing 
system. 

In another example of a manual system, commonly assigned U.S. 

25 Patent No. 5,61 9,738, entitled "Pre-processing Image Editing" filed by Petruchik 
et al., describes a film camera that includes electronic display. The electronic 
display is provided with the driver that the presents movable markers to frame and 
edit the electronically displayed image. Electronic manipulation of the markers 
provides for zooming and cropping the image, as well as changing the image 

30 format and recording horizontal or vertical orientation. The control that records 
editing data on the film in a predetermined position relative to the exposed scene 
image. The camera includes an input for selecting print format from a 
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predetermined set of print formats, each having a predetermined size and aspect 
ratio, and control limits the frame formats to those that are consistent with the 
selecting print format. A variety of controls and user inputs facilitate the process 
of selecting and forming the image. These controls permit the camera user to 
5 have great flexibility in editing. 

Similarly commonly assigned U.S. Pat. No. 5,898,779, entitled 
"Photographic System With Selected Area Image Authentication" filed by Squilla 
et al. on April 14, 1997, describes a public key encryption system for 
authenticating an image. The encryption system comprises means for generating 

1 0 one or more patterns each composed of at least one individual area that is visible 
together with the image of the object, means for designating at least one individual 
area as an active area of the image suitable for authentication and for generating 
location data identifying the active area. In one form, a single grid pattern can be 
used in which a tile area designation switch is used for cycling through the tile 

1 5 areas and highlighting one tile area after the other to form a user customizable tile 
pattern. The tile pattern is composed of a number of intersecting lines which 
define the individual tile areas, one or more of which may be designated as an 
active area for purposes of securing the image. In the course of deciding which tile 
areas should become active areas, one or more tile areas are highlighted for 

20 consideration. Alternatively, a tile pattern selection switch can be used to select a 
particular preformed tile pattern having a preformed highlighted area indicating 
the area to be used. A security service switch is used to designate a particular 
arrangement of highlighted tile areas or a preformed tile pattern. 

In still another example of a manual system, commonly assigned 

25 U.S. Patent Application No. 10/292,235, entitled "User Interface for Controlling 
Cropping In An Electronic Camera", filed by Brost on November 12, 2002, 
describes a camera system that captures an image and presents a light image based 
upon the captured image. A cropper is disposed in the camera and is switchable 
among a plurality of settings. Each setting has a different rectangular cross- 

30 sectioned window in the viewfinder. A cropping controller is operatively 

connected to the capture unit and to the cropper. The cropping control has a 
cropping input element that is accessible external to the body. The cropping input 
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element is movable between first and second opposed cropping control positions 
to change the settings. The cropping control positions define a cropping control 
axis that is perpendicular to an optical axis of the capture unit. This approach 
provides a simplified control structure to allow adjustment of borders for use in 
5 cropping an image and provides a useful and commercially viable way to select 
from among a rage of cropped images, each centered on a scene image. 

In another example of this type, cameras that comply with the 
Advanced Photographic System standards are typically capable of capturing 
images using a first format having an aspect ratio that corresponds to the aspect 

10 ratio of a high definition television and that allow a user to record a signal on the 
film indicating that a print made from the captured image is to be rendered in the 
high definition format, in a so-called classic format or a so-called panoramic 
format. When the classic or panoramic formats are selected, the photofinisher 
uses a cropped portion of the captured image each centered with respect to the 

1 5 high definition image and each having a different aspect ratio. 

What is needed therefore is a display device such as a digital or 
hybrid camera, Personal Digital Assistant or any other system capable of 
presenting an electronic image that also has a user interface system that enables 
rapid and simplified manual designation of an area of importance within a 

20 captured image to provide an output from which it can easily be determined which 
portion of the image has been designated. 

It will be appreciated that any system that enables even a modest a 
reduction in the number of key strokes or user input actions required to execute 
such a designation is particularly useful because it makes it more likely that a user 

25 will make a designation. Accordingly, a further need exists for a display device 
and a method for enabling a manual designation of an area of importance in an 
image that is adapted to reduce the overall amount of effort required by the user of 
a display device to identify an area of importance in the image. 

Still another need is for a system that presents evaluation images 

30 that allow a user to quickly and more easily to see what is contained in a portion 
of the image being considered by the user for use in determining an area of 
importance. 
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SUMMARY OF THE INVENTION 
In one aspect of the invention, a display device is provided. A 
display device has a source of an original image and a display. A user input 
system is adapted to generate a non-directional signal in response to a user input 
5 action. A controller is provided and is adapted to determine a set of portions of 
the original image, each portion including less than all of the original image and 
with the set having at least one portion that is non-central with respect to the 
original image and to successively designate a different one of a set of portions of 
the original image in response to each non-directional signal. The controller is 

1 0 further adapted to cause the display to present a portion evaluation image showing 
the currently designated portion of the original image and to determine an area of 
importance in the original image based upon the currently designated portion. 
Wherein each portion evaluation image shows the currently designated portion 
having a magnification that is greater than the magnification that the currently 

15 designated portion has when presented as a part of the original image. 

In another aspect of the invention, a display device is provided. 
The display device has a source of an original image, a display and a user input 
system adapted to generate an advance signal and a save signal. A controller is 
provided and is adapted to detect the advance signal and, in response thereto, to 

20 cause the display to present a sequence of portion evaluation images each 

representing the image content of one of a set of different portions of the original 
image, with the predetermined set of portions including at least one portion that is 
non-central with respect to the original image. Wherein the controller determines 
an area of importance in the original image based upon the portion of the original 

25 image presented when the controller detects the save signal. 

In yet another aspect of the invention, a display device is provided. 
The display device has a source of an original image, a display, and a user input 
system adapted to generate and non-directional advance signal in response to a 
user input action. A controller is adapted to detect the non-directional advance 

30 signal and, in response thereto, to define a number of portion evaluation images 
each comprising image information from a portion of the original image with each 
portion being located relative to a predefined anchor point within the original 
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image. The controller is further adapted to cause a different portion evaluation 
image to be presented on the display in response to each advance signal. The 
controller is additionally adapted to determine, from the non-directional advance 
signal, a user designation of a portion of the original image and to use the 
5 designation to determine an area of importance in the original image. Wherein at 
least one anchor point is located so that at least one portion is non-central with 
respect to the original image. 

In a further aspect of the invention, a method is provided for 
operating a display device. In accordance with the method, an original image is 

1 0 obtained and an evaluation image is presented having an appearance that 

corresponds to the original image. A set of different portions is defined in the 
original image with each portion comprising less than all of the original image and 
at least one of the portions being non-central with the original image. A non- 
directional user input action is detected during presentation of the evaluation 

1 5 image. One of the set of portions is designated in response to each detected non- 
directional user input action, and a portion evaluation image is presented that 
corresponds to the designated portion with the portion evaluation image showing 
the currently designated portion having a magnification that is greater than the 
magnification that the currently designated portion has when the currently 

20 designated portion is presented as a part of the original image. An area of 
importance is determined based upon the designated portion. 

In still another aspect of the invention, a method for operating a 
display device is provided. In accordance with the method, an original image is 
obtained and an evaluation image of the original image is displayed. An advance 

25 a user input action is detected that does not include a directional input relative to 
the displayed evaluation image. At least one portion of a set of different portions 
of the original image is selected based upon the user input action. A portion 
evaluation image is presented that indicates the image information in the original 
image that is contained within the currently designated portion. A save user input 

30 action is detected and an area of importance is determined based upon the selected 
portion displayed when the user save input user action is detected. Wherein at 
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least one of the predetermined set of portions of the original image is non-central 
with respect to the original image. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is an illustration of one simplified block diagram of one 
5 exemplary embodiment of a display device according to the present invention; 

Fig. 2 is an illustration of a back view of the display device of Fig. 

i; 

Fig. 3 is a flow diagram of one embodiment of the invention; 
Fig. 4 is an illustration of an original image with one arrangement 
1 0 of portions associated therewith; 

Fig. 5 is an illustration of an original image with another 
arrangement of portions associated therewith; 

Fig. 6 shows one embodiment of a portion evaluation image; 
Fig. 7 shows another embodiment of a portion evaluation image; 
1 5 Fig. 8 shows still another embodiment of a portion evaluation 

image; 

Fig. 9 illustrates an area of importance based upon a designated 
portion of an original image; 

Fig. 10 shows another embodiment of a display device; 
20 Fig. 11 shows a flow diagram of another embodiment of a method 

for determining an area of importance; 

Fig. 12 shows another embodiment of a display device; 

Fig. 13 shows another embodiment of a method for determining an 
area of importance; 

25 Fig. 14 shows another embodiment of a display device showing an 

original image with anchor points; and 

Fig. 1 5 shows another embodiment of a display device showing a 
portion evaluation image with optional differently sized portion shapes. 

DETAILED DESCRIPTION OF THE INVENTION 
30 Fig. 1 shows a block diagram of an embodiment of a display device 

10. Fig. 2 shows a back, elevation view of the display device 10 of Fig. 1. As is 
shown in Figs. 1 and 2, display device 10 takes the form of a digital camera 12 
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comprising a body 20 containing an image capture system 22 having a lens system 
23, an image sensor 24, a signal processor 26, an optional display driver 28 and a 
display 30. In operation, light from a scene is focused by lens system 23 to form 
an image on image sensor 24. Lens system 23 can have one or more elements. 
5 Lens system 23 can be of a fixed focus type or can be manually or 

automatically adjustable. In the embodiment shown in Fig. 1, lens system 23 is 
automatically adjusted. Lens system 23 can be simple, such as having a single 
focal length with manual focusing or a fixed focus. In the example embodiment 
shown in Fig. 1, taking lens unit 22 is a motorized 6x zoom lens unit in which a 

10 mobile element or elements (not shown) are driven, relative to a stationary 

element or elements (not shown) by lens driver 25. Lens driver 25 controls both 
the lens focal length and the lens focus position of lens system 23 and sets a lens 
focal length and/or position based upon signals from signal processor 26, an 
optional automatic range finder system 27, and/or controller 32. 

15 The focal length and/or focus position of lens system 23 can be 

automatically selected using a variety of known strategies. For example, in one 
embodiment, image sensor 24 is used to provide multi-spot autofocus using what 
is called the "through focus" or "whole way scanning" approach. In such an 
approach the scene is divided into a grid of regions or spots, and the optimum 

20 focus distance is determined for each image region. The optimum focus distance 
for each region is determined by moving lens system 23 through a range of focus 
distance positions, from the near focus distance to the infinity position, while 
capturing images. Depending on the design of digital camera 12, between four 
and thirty-two images may need to be captured at different focus distances. 

25 Typically, capturing images at eight different distances provides suitable 
accuracy. 

The captured image data is then analyzed to determine the 
optimum focus distance for each image region. This analysis begins by band-pass 
filtering the sensor signal using one or more filters, as described in commonly 
30 assigned U.S. Patent No. 5,874,994 "Filter Employing Arithmetic Operations for 
an Electronic Synchronized Digital Camera" filed by Xie et al., on December 11, 
1995, the disclosure of which is herein incorporated by reference. The absolute 
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value of the bandpass filter output for each image region is then peak detected, in 
order to determine a focus value for that image region, at that focus distance. 
After the focus values for each image region are determined for each captured 
focus distance position, the optimum focus distances for each image region can be 
5 determined by selecting the captured focus distance that provides the maximum 
focus value, or by estimating an intermediate distance value, between the two 
measured captured focus distances which provided the two largest focus values, 
using various interpolation techniques. 

The lens focus distance to be used to capture a digital image can 

10 now be determined. In a preferred embodiment, the image regions corresponding 
to a target object (e.g. a person being photographed) are determined. The focus 
position is then set to provide the best focus for these image regions. For 
example, an image of a scene can be divided into a plurality of sub-divisions. A 
focus evaluation value representative of the high frequency component contained 

1 5 in each subdivision of the image can be determined and the focus evaluation 
values can be used to determine object distances as described in commonly 
assigned U.S. Patent No. 5,877,809 entitled "Method Of Automatic Object 
Detection In An Image", filed by Omata et al. on October 15, 1996, the disclosure 
of which is herein incorporated by reference. If the target object is moving, object 

20 tracking may be performed, as described in commonly assigned U.S. Patent No. 
6,067,1 14 entitled "Detecting Compositional Change in Image" filed by Omata et 
al. on October 26, 1996, the disclosure of which is herein incorporated by 
reference. In an alternative embodiment, the focus values determined by "whole 
way scanning" are used to set a rough focus position, which is refined using a fine 

25 focus mode, as described in commonly assigned U.S. Patent No. 5,715,483, 

entitled "Automatic Focusing Apparatus and Method", filed by Omata et al. on 
October 1 1 , 1998, the disclosure of which is herein incorporated by reference. 

In one embodiment, bandpass filtering and other calculations used 
to provide auto-focus information for digital camera 12 are performed by digital 

30 signal processor 26. In this embodiment, digital camera 12 uses a specially 
adapted image sensor 24, as is shown in commonly assigned U.S. Patent No 
5,668,597 entitled "An Electronic Camera With Rapid Automatic Focus Of An 
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Image Upon A Progressive Scan Image Sensor", filed by Parulski et al. on 
December 30, 1994, the disclosure of which is herein incorporated by reference, 
to automatically set the lens focus position. As described in the f 597 patent, only 
some of the lines of sensor photoelements (e.g. only % of the lines) are used to 
5 determine the focus. The other lines are eliminated during the sensor readout 
process. This reduces the sensor readout time, thus shortening the time required 
to focus lens system 23 

In an alternative embodiment, digital camera 12 uses a separate 
optical or other type (e.g. ultrasonic) of rangefinder 27 to identify the subject of 

10 the image and to select a focus position for lens system 23 that is appropriate for 
the distance to the subject. Rangefinder 27 can operate lens driver 25, directly or 
as shown in Fig. 1 , can provide signals to signal processor 26 or controller 32 
from which signal processor 26 or controller 32 can generate signals that are to be 
used for image capture. A wide variety of suitable multiple sensor rangefinders 

1 5 27 known to those of skill in the art are suitable for use. For example, U.S. Patent 
No. 5,440,369 entitled "Compact Camera With Automatic Focal Length 
Dependent Exposure Adjustments" filed by Tabata et al. on November 30, 1993, 
the disclosure of which is herein incorporated by reference, discloses one such 
rangefinder 27. The focus determination provided by rangefinder 27 can be of the 

20 single-spot or multi-spot type. Preferably, the focus determination uses multiple 
spots. In multi-spot focus determination, the scene is divided into a grid of areas 
or spots, and the optimum focus distance is determined for each spot. One of the 
spots is identified as the subject of the image and the focus distance for that spot is 
used to set the focus of lens system 23. 

25 A feedback loop is established between lens driver 25 and camera 

controller 32 so that camera controller 32 can accurately set the focus position of 
lens system 23. 

Lens system 23 is also optionally adjustable to provide a variable 
zoom. In the embodiment shown lens driver 25 automatically adjusts the position 
30 of one or more mobile elements (not shown) relative to one or more stationary 

elements (not shown) of lens system 23 based upon signals from signal processor 
26, an automatic range finder system 27, and/or controller 32 to provide a zoom 



- 12- 



magnification. Lens system 23 can be of a fixed magnification, manually 
adjustable and/or can employ other known arrangements for providing an 
adjustable zoom. 

Light from the scene that is focused by lens system 23 onto image 
5 sensor 24 is converted into image signals representing an image of the scene. 
Image sensor 24 can comprise a charge couple device (CCD), a complimentary 
metal oxide sensor (CMOS), or any other electronic image sensor known to those 
of ordinary skill in the art. The image signals can be in digital or analog form. 

Signal processor 26 receives image signals from image sensor 24 

10 and transforms the image signals into an image in the form of digital data. The 

digital image can comprise one or more still images, multiple still images and/or a 
stream of apparently moving images such as a video segment. Where the digital 
image data comprises a stream of apparently moving images, the digital image 
data can comprise image data stored in an interleaved or interlaced image form, a 

1 5 sequence of still images, and/or other forms known to those of skill in the art of 
digital video. 

Signal processor 26 can apply various image processing algorithms 
to the image signals when forming a digital image. These can include but are not 
limited to color and exposure balancing, interpolation and compression. Where 

20 the image signals are in the form of analog signals, signal processor 26 also 
converts these analog signals into a digital form. 

Controller 32 controls the operation the display device 10 during 
imaging operations, including but not limited to image capture system 22, display 
30 and memory such as memory 40. Controller 32 causes image sensor 24, signal 

25 processor 26, display 30 and memory 40 to capture and store original images in 
response to signals received from a user input system 34, data from signal 
processor 26 and data received from optional sensors 36. Controller 32 can 
comprise a microprocessor such as a programmable general purpose 
microprocessor, a dedicated micro-processor or micro-controller, a combination of 

30 discrete components or any other system that can be used to control operation of 
display device 10. 
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Controller 32 cooperates with a user input system 34 to allow 
display device 10 to interact with a user. User input system 34 can comprise any 
form of transducer or other device capable of receiving an input from a user and 
converting this input into a form that can be used by controller 32 in operating 
5 display device 10. For example, user input system 34 can comprise a touch screen 
input, a touch pad input, a 4-way switch, a 6-way switch, an 8-way switch, a 
stylus system, a trackball system, a joystick system, a voice recognition system, a 
gesture recognition system or other such systems. In the digital camera 12 
embodiment of display device 10 shown in Figs. 1 and 2 user input system 34 

10 includes a shutter trigger button 60 that sends a trigger signal to controller 32 
indicating a desire to capture an image. 

In the embodiment shown in Figs. 1 and 2, user input system 34 
also includes a wide-angle zoom button 62 and a tele-zoom button 64 that 
cooperate with controller 32 to control the zoom settings of lens system 23 

1 5 causing lens system 23 to zoom out when wide angle zoom button 62 is depressed 
and to zoom in when tele-zoom button 64 is depressed. Wide-angle zoom lens 
button 62 and telephoto zoom button 64 can also be used to provide signals that 
cause signal processor 26 to process image signal so that the digital image formed 
thereby appears to have been captured at a different zoom setting than that 

20 actually provided by the optical lens system. This can be done by using a subset 
of the image signals from image sensor 24 and interpolating the subset of the 
image signals to form the digital image. User input system 34 can also include 
other buttons including the edit button 66 shown in Fig. 2, the function of which 
will be described in greater detail below. 

25 Sensors 36 are optional and can include light sensors and other 

sensors known in the art that can be used to detect conditions in the environment 
surrounding display device 10 and to convert this information into a form that can 
be used by controller 32 in governing operation of display device 10. Sensors 36 
can also include biometric sensors adapted to detect characteristics of a user for 

30 security and affective imaging purposes. 

Controller 32 causes an image signal and corresponding digital 
image to be formed when a trigger condition is detected. Typically, the trigger 
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condition occurs when a user depresses shutter trigger button 60, however, 
controller 32 can determine that a trigger condition exists at a particular time, or at 
a particular time after shutter trigger button 60 is depressed. Alternatively, 
controller 32 can determine that a trigger condition exists when optional sensors 
5 36 detect certain environmental conditions. 

Controller 32 can also be used to generate metadata in association 
with each image. Metadata is data that is related to a digital image or a portion of 
a digital image but that is not necessarily observable in the image itself. In this 
regard, controller 32 can receive signals from signal processor 26, camera user 

10 input system 34 and other sensors 36 and, optionally, generates metadata based 
upon such signals. The metadata can include but is not limited to information 
such as the time, date and location that the original image was captured, the type 
of image sensor 24, mode setting information, integration time information, taking 
lens unit setting information that characterizes the process used to capture the 

15 original image and processes, methods and algorithms used by display device 10 
to form the original image. The metadata can also include but is not limited to 
any other information determined by controller 32 or stored in any memory in 
display device 10 such as information that identifies display device 10, and/or 
instructions for rendering or otherwise processing the digital image with which the 

20 metadata is associated. The metadata can also comprise an instruction to 
incorporate a particular message into digital image when presented. Such a 
message can be a text message to be rendered when the digital image is presented 
or rendered. The metadata can also include audio signals. The metadata can 
further include digital image data. The metadata can also include any other 

25 information entered into display device 10. 

The digital images and optional metadata, can be stored in a 
compressed form. For example where the digital image comprises a sequence of 
still images, the still images can be stored in a compressed form such as by using 
the JPEG (Joint Photographic Experts Group) ISO 10918-1 (ITU - T.81) standard. 

30 This JPEG compressed image data is stored using the so-called "Exif ' image 

format defined in the Exchangeable Image File Format version 2.2 published by 
the Japan Electronics and Information Technology Industries Association JEITA 
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CP-3451. Similarly, other compression systems such as the MPEG-4 (Motion 
Pictures Export Group) or Apple QuickTime ™ standard can be used to store 
digital image data in a video form. Other image compression and storage forms 
can be used. 

5 The digital images and metadata can be stored in a memory such as 

memory 40. Memory 40 can include conventional memory devices including 
solid state, magnetic, optical or other data storage devices. Memory 40 can be 
fixed within display device 10 or it can be removable. In the embodiment of Fig. 
1, display device 10 is shown having a memory card slot 46 that holds a 

10 removable memory 48 such as a removable memory card and has a removable 

memory interface 50 for communicating with removable memory 48. The digital 
images and metadata can also be stored in a remote memory system 52 that is 
external to display device 10 such as a personal computer, computer network or 
other imaging system. 

15 In the embodiment shown in Figs. 1 and 2, display device 10 has a 

communication module 54 for communicating with remote memory system 52. 
The communication module 54 can be for example, an optical, radio frequency or 
other transducer that converts image and other data into a form that can be 
conveyed to the remote display device by way of an optical signal, radio 

20 frequency signal or other form of signal. Communication module 54 can also be 
used to receive a digital image and other information from a host computer or 
network (not shown). Controller 32 can also receive information and instructions 
from signals received by communication module 54 including but not limited to, 
signals from a remote control device (not shown) such as a remote trigger button 

25 (not shown) and can operate display device 10 in accordance with such signals. 

Signal processor 26 and/or controller 32 also use image signals or 
the digital images to form evaluation images which have an appearance that 
corresponds to original images stored in display device 10 and are adapted for 
presentation on display 30. This allows users of display device 10 to use a display 

30 such as display 30 to view images that correspond to original images that are 

available in display device 10. Such images can include, for example images that 
have been captured by image capture system 22, and/or that were otherwise 
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obtained such as by way of communication module 54 and stored in a memory 
such as memory 40 or removable memory 48. 

Display 30 can comprise, for example, a color liquid crystal display 
(LCD), organic light emitting display (OLED) also known as an organic electro- 
5 luminescent display (OELD) or other type of video display. Display 30 can be 
external as is shown in Fig. 2, or it can be internal for example used in a 
viewfinder system 38. Alternatively, display device 10 can have more than one 
display 30 with, for example, one being external and one internal. 

Signal processor 26 and/or controller 32 can also cooperate to 

10 generate other images such as text, graphics, icons and other information for 
presentation on display 30 that can allow interactive communication between 
controller 32 and a user of display device 10, with display 30 providing 
information to the user of display device 10 and the user of display device 10 
using user input system 34 to interactively provide information to display device 

15 10. Display device 10 can also have other displays such as a segmented LCD or 
LED display (not shown) which can also permit signal processor 26 and/or 
controller 32 to provide information to user 10. This capability is used for a 
variety of purposes such as establishing modes of operation, entering control 
settings, user preferences, and providing warnings and instructions to a user of 

20 display device 10. Other systems such as known systems and actuators for 

generating audio signals, vibrations, haptic feedback and other forms of signals 
can also be incorporated into display device 10 for use in providing information, 
feedback and warnings to the user of display device 10. 

Typically, display 30 has less imaging resolution than image sensor 

25 24. Accordingly, signal processor 26 reduces the resolution of image signal or 

digital image when forming evaluation images adapted for presentation on display 
30. Down sampling and other conventional techniques for reducing the overall 
imaging resolution can be used. For example, resampling techniques such as are 
described in commonly assigned U.S. Patent No. 5,164,831 "Electronic Still 

30 Camera Providing Multi-Format Storage Of Full And Reduced Resolution 

Images" filed by Kuchta et al. on March 1 5, 1990, can be used. The evaluation 
images can optionally be stored in a memory such as memory 40. The evaluation 
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images can be adapted to be provided to an optional display driver 28 that can be 
used to drive display 30. Alternatively, the evaluation images can be converted 
into signals that can be transmitted by signal processor 26 in a form that directly 
causes display 30 to present the evaluation images. Where this is done, display 
5 driver 28 can be omitted. 

Display device 10 can obtain original images for processing in a 
variety of ways. For example, display device 10 can capture an original image 
using an image capture system 22 as described above. Imaging operations that 
can be used to obtain an original image using image capture system 22 include a 

10 capture process and can optionally also include a composition process and a 
verification process. 

During the optional composition process, controller 32 optionally 
provides an electronic viewfinder effect on display 30. In this regard, controller 
32 causes signal processor 26 to cooperate with image sensor 24 to capture 

1 5 preview digital images during composition and to present a corresponding 
evaluation images on display 30. 

In the embodiment shown in Figs. 1 and 2, controller 32 enters the 
image composition process when shutter trigger button 60 is moved to a half 
depression position. However, other methods for determining when to enter a 

20 composition process can be used. For example, one of user input system 34, for 
example, the edit button 68 shown in Fig. 2 can be depressed by a user of display 
device 10, and can be interpreted by controller 32 as an instruction to enter the 
composition process. The evaluation images presented during composition can 
help a user to compose the scene for the capture of an original image. 

25 The capture process is executed in response to controller 32 

determining that a trigger condition exists. In the embodiment of Figs. 1 and 2, a 
trigger signal is generated when shutter trigger button 60 is moved to a full 
depression condition and controller 32 determines that a trigger condition exists 
when controller 32 detects the trigger signal. During the capture process, 

30 controller 32 sends a capture signal causing signal processor 26 to obtain image 
signals from image sensor 24 and to process the image signals to form digital 
image data comprising an original digital image. 
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During the verification process, an evaluation image corresponding 
to the original digital image is optionally formed for presentation on display 30 by 
signal processor 26 based upon the image signal. In one alternative embodiment, 
signal processor 26 converts each image signal into a digital image and then 
5 derives the corresponding evaluation image from the original digital image. The 
corresponding evaluation image is supplied to display 30 and is presented for a 
period of time. This permits a user to verify that the digital image has a preferred 
appearance. 

Original images can also be obtained by display device 10 in ways 

1 0 other than image capture. For example, original images can by conveyed to 

display device 10 when such images are recorded on a removable memory that is 
operatively associated with memory interface 50. Alternatively, original images 
can be received by way of communication module 54. For example, where 
communication module 54 is adapted to communicate by way of a cellular 

1 5 telephone network, communication module 54 can be associated with a cellular 
telephone number or other identifying number that for example another user of the 
cellular telephone network such as the user of a telephone equipped with a digital 
camera can use to establish a communication link with display device 10 and 
transmit images which can be received by communication module 54. 

20 Accordingly, there are a variety of ways in which display device 10 can receive 
images and therefore it is not essential that display device 10 have an image 
capture system so long as other means such as those described above are available 
for importing images into display device 1 0. 

Fig. 3 shows a flow diagram depicting a first embodiment of a 

25 method for determining an area of importance in an archival image using, for 

example, display device 10. In accordance with the method, an original image is 
obtained (step 1 00). This can be done by capture or by importing the original 
image as discussed above, using other conventional archival image capture or 
downloading techniques. For example, as discussed above, an original image can 

30 be any digital image that is captured by image capture system 22, downloaded 
form a removable memory 48, or otherwise obtained by communication module 
54. 
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After the original image is obtained, an evaluation image is 
presented (step 102). Under certain circumstances, where, for example, the 
imaging capabilities of display 30 has sufficient image resolution, the evaluation 
image can comprise the original image. In circumstances where the original 
5 image provides more image information than can be presented using display 30, 
controller 32 can cause the formation of an evaluation image by down sampling or 
otherwise adapting the original image to form an evaluation image to based upon 
the original image as described above. Such an evaluation image has an 
appearance that corresponds to the appearance of the original image and is 

1 0 adapted for presentation using display 30. 

It is then determined whether a user input action has occurs (step 
1 04). Typically, the user input action comprises some action that can be sensed 
by user input system 34. This can take a variety of forms. For example, the user 
input action can comprise pressing edit button 68 shown in Fig. 2. In this 

15 example, when edit button 68 is pressed user input system 34 generates an edit 
signal that is received by controller 32. When controller 32 receives the edit 
signal, controller 32 determines that a user input action has occurred. The user 
input action can take a wide variety of other forms only some of which will be 
described in greater detail later herein and, user input system 34 and controller 32 

20 will be adapted to cooperate with such different forms of user input actions. 

In the embodiment of Fig. 3, when a user input action is detected, 
controller 32 designates at least one portion of a set of portions of the original 
image (step 106). In this regard, controller 32, optionally in cooperation with 
signal processor 34, defines a set of different portions within the original image. 

25 Each portion comprises at least some of the original image. The sizes, locations, 
aspect ratios, and shapes of each portion within the set can vary. At least one of 
the portions can be centered along a non-central portion of the original image. A 
set of portions can incorporate a variety of different portions. In one embodiment, 
between 3 and 30 different portions are provided, in another embodiment, nine 

30 different portions are provided, and in still other embodiments one hundred or 
more portions can be incorporated in the set. 
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The set of portions can comprise portions that take on a variety of 
similar or disparate shapes and sizes. For example, in one embodiment, all of the 
portions can comprise an area which can incorporate for example, 75 percent of 
the image space provided by the original image and can have the same aspect ratio 
5 as the original image. However, the multiple portions differ from each other 
based upon where the portions are located relative to the original image. For 
example, a first portion can be centered with respect to the original image while at 
least one other portion defines an area of the original image that is not centered 
with respect to the original image. For example, one such non-central predefined 

1 0 portion can be centered on the original image along the horizontal axis of the 
original image but may be shifted vertically for example downward, so that the 
lower edge of the portion lays on top of the lower edge of the evaluation copy of 
the original image. 

Additionally, a portion of an original image can be defined in any 

1 5 way that the designers of display device 1 0 believe to be useful to a user 
designating a portion of the original image as an area of the importance. 
Although, in the above described example, the portions have been described as 
being identical in size, but varied in location, is also possible that the portions 
could vary in size, aspect ratio, shape, or in combinations of these or other 

20 parameters. 

The set of portions can also be defined based upon analysis of the 
original image performed in an image analysis step (not shown). In one example 
embodiment of this type an original image can be analyzed using known 
techniques such as frequency analysis to identify areas of the original image that 

25 are in focus and can define portions that correspond to different ones of the in 
focus area. In another embodiment, where display device 10 comprises a digital 
camera, the original image can be separated into portions based upon focus 
distance information obtained, for example, during an autofocus process. More 
complex scene image analysis techniques including, but not limited to, edge 

30 detection, illumination pattern recognition, and focus identification techniques can 
be applied to the original image by controller 32 and/or signal processor 34 to 
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identify useful portions of the original image, such as potential subject areas 
within the original image. 

Fig. 4 shows one example of illustration of a set of portions of an 
original image 120. As is shown in Fig. 4, original image 120 is divided into 
5 portions 122, 124, 126, 128, 130, 132, 134, 136, and 138. It will be appreciated 
that, in this example, each of the portions is separate from the other portions. 
However, this is not necessary and, the portions can at least in part overlap, as is 
illustrated in Fig. 5 which shows original image 120 having a set of six portions 
140, 142, 144, 146, 148 and 150 some of which overlap. As is also illustrated in 

10 Fig. 5, the portions can be shaped differently. 

A first portion from the set of portions is automatically designated 
by controller 32 when the user input action is detected (step 106). In one 
embodiment of the invention, controller 32 is adapted to designate portions in the 
set of portions using a predetermined sequence. For example, when a user of 

15 display device 10 first depresses edit button 68, controller 32 can designate a first 
portion from the set of portions of in accordance with the predetermined sequence 
designated for the set of portions. However, the sequence and order of 
designation is not critical and can vary. 

Controller 32 and, optionally, signal processor 34, cause a portion 

20 evaluation image to be presented for the designated portion (step 108). The 

portion evaluation image indicates the image information in the original image 
that is contained within the currently designated portion. 

A portion evaluation image can be formed in a variety of ways. 
One form of a portion evaluation image 160 is shown in Fig. 6, this type of 

25 portion evaluation image comprises an image that includes only image 

information from the currently designated portion such as portion 138 of Fig. 4. 
An advantage of this type of portion evaluation image is that it shows the 
designated portion having an appearance that is magnified as compared to the 
appearance of the designated portion when the designated portion is presented on 

30 display 30 together with other image elements of the original image. Because of 
this, a user who observes the presented portion evaluation image can have an 
opportunity to view the designated portion in greater detail and can thereby detect 
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errors or other image conditions that the user may find undesirable in the image 
and that may not be readily apparent in the designated portion when the 
designated portion is presented together with other image information from the 
original image. 

5 An alternate embodiment of a method for presenting a portion 

evaluation image 162 is shown in Fig. 7. As is shown in Fig. 7, in this 
embodiment, a portion evaluation image 1 62 is provided having all of the image 
information from original image 1 20 with portions that are outside of a currently 
designated portion 164 being masked by mask 166. In the embodiment shown, 

10 mask 166 comprises an opaque region superimposed over non-designated portions 
of the evaluation image. However, it is not necessary that mask 166 is fully 
opaque and, in practice, mask 1 66, can comprise any form of alteration to the 
appearance of an evaluation image 160 that can be used to demarcate or to 
visually separate a designated portion 164 from other portions of evaluation image 

15 1 60. For example, the appearance of the designated portions of evaluation image 
160 can be adjusted to appear without color, with modified color or can appear 
with obviously adjusted sharpness. 

In still another embodiment, shown in Fig. 8, a designated portion 
such as portion 138 of original image 120 is shown in a portion evaluation image 

20 162 as being separated from other portions of evaluation image 160 presented on 
display 30 having substantially all of the image information from original image 
120 using a separator 168 such as a line. The portion evaluation image can take 
other forms. 

Returning onto Fig. 3, during or after the presentation of a portion 
25 evaluation image that shows a currently designated portion, controller 32 monitors 
user input system 34 to detect a user reset action (step 110). A user reset action 
can comprise any form of user action that can be detected by user input system 34 
and that provides a reset signal to controller 32. When controller 32 detects such a 
reset signal, controller 32 determines that a currently designated portion of the 
30 original image is not to be considered to be the designated portion. The user reset 
action can be an instruction to user input system 34 indicating that the user wishes 
to select a different portion. Alternatively, the user reset action can comprise a 
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signal indicating that the user does not wish to designate any portion of the 
original image. Where a user elects not to designate a portion, the designating 
process ends. 

Where a user does not enter a reset action controller 32 determines 
5 an area importance based upon the designation (step 112). In one embodiment, 
the area of importance determination is made by determining that the area in the 
designated portion comprises the area of importance. In another embodiment, the 
area importance determination can be made by defining the original image for use 
as an area importance based upon the designated portion. For example, where 

10 portion 148 of Fig. 5 is selected as the designated portion, controller 32 can 

determine that the area of importance comprises an area that includes the area of 
importance designated portion, but is not necessarily coextensive therewith. In 
this example, as is shown in Fig. 9, the designated area can comprise, for example, 
an area importance 1 70 having a different shape or aspect ratio that is located 

1 5 within an original image 120 based upon the location of the determined designated 
portion shown in this example as portion 148. It will be appreciated that this can 
be done in a variety of ways. 

After an area importance is determined for the original image, the 
area importance determination is used by controller 32 for some purpose (step 

20 1 14). In one embodiment, controller 32 is adapted to generate area of importance 
data based upon determined area importance and to associate the generated area of 
importance data with the original image. The area importance data can comprise, 
any form of data or other encodement of information from which a user of the 
original image can determine which portion of the original image was designated 

25 to be important by the user. 

The area of importance data can be associated with the original 
image in a variety of ways. For example, area importance data can be stored as 
metadata in a digital image that is formed based upon the original image, or in 
metadata that is stored with the original image itself. In cases where display 

30 device 10 incorporates an electronic image capture unit and a film image capture 
unit, the area of importance data can be stored with an original image recorded on 
the film for example using optical or magnetic encodements. 
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In still another embodiment, controller 32 can associate the area of 
importance data with the original image by generating a revised image based upon 
image information from the area of importance. This revised image can be stored 
in association with the original image or some image other image derived 
5 therefrom. The revised image can be also be stored, without preserving the 
original image. This latter alternative has the advantage of conserving memory 
space inside a display device 10 and in certain implementations it can reduce the 
amount of data required to store the desired image by using a only a fraction of the 
image information in the original image or by adjusting compression parameters 

10 used in forming an image for storage based upon the original image by using only 
modest degrees of compression for image elements that are within the area 
importance of the original image and by using more extensive degrees of 
compression for storing image information from areas that are not within the area 
importance in the original image. 

1 5 Controller 32 can also use the determined area of importance for 

other purposes. For example, controller 32 can use the determined area of 
importance in making determinations about post capture image editing and 
modification. For example, the white balance and/or exposure settings used by 
controller 32 and/or signal processor 34 to form the original image can be adjusted 

20 electronically based upon analysis of the determined area of importance. Further, 
controller 32 and/or signal processor 34 can determine a variety of camera settings 
based upon conditions determined in the area importance. The determined area of 
importance can then be used to edit and modify the original image, such as for 
image cropping and similar such functions, to define capture parameters for 

25 subsequent image capture, or to provide information in the form of metadata that 
can be associated with the original image for use in later processing and 
evaluation, analysis, and/or use of the original image. 

Figs. 10 and 1 1 show another embodiment of a display device 10 
and method of the present invention. In embodiment of Figs. 10 and 11, the user 

30 designation can be performed using only one input of a user input system 34 
designated, in this embodiment, as edit button 68 which is used to designate a 
portion of the original image. As is shown in the embodiment of Fig. 11, after an 
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original image has been obtained (step 1 82) an evaluation image based upon the 
obtained original image is presented (step 184) using for example display 30. 

Controller 32 monitors user input system 34 to detect whether a 
user of display device 10 makes a user input action during presentation of the 
5 evaluation image. Where controller 32 detects that a user makes the user input 
action of depressing edit button 68 of display device 10 during presentation of the 
evaluation image (step 186), controller 32 will cause a first current portion 
evaluation image to be presented on display 30 (step 1 88). The first current 
portion evaluation image can be a portion evaluation image based upon any of the 
10 predefined portions. In one embodiment the current portion evaluation image 
comprises a portion evaluation image that is concentric with the original image, 
and therefore has the appearance of a zoomed image. However, this is not 
necessary and the initial portion evaluation image can comprise any portion of the 
original image. 

15 If the user does not make a user input action within the timeout 

period (step 192), controller 32 can determine from this that the user does not 
wish to designate any portion of the original image as an area of importance (step 
194). Alternatively, controller 32 can interpret the absence of a user input action 
within the timeout period (not shown) to mean that the user selects the entire 

20 original image as an area of importance. 

If controller 32 detects that a user input action that continues after 
an advance time period, (e.g. where edit button 68 is still depressed after 3 
seconds) (step 1 90), controller 32 will designate a different portion from the set of 
predetermined portions of as a current portion of the original image (step 1 96) and 

25 will cause a portion evaluation image to be presented on display 30 using the 
currently designated portion (step 198). 

Controller 32 is adapted to detect whether a user input action is 
performed during presentation of the current portion evaluation image (step 200). 
Where a user input action is detected during presentation of the current portion 

30 evaluation image, controller 32 determines whether the user continues the input 
action for at least an advance time (step 202). For example, controller 32 can 
detect whether user interaction is sustained for an advance time period of, for 
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example, three seconds. Where this condition is detected, controller 32 causes a 
different portion of the original image to be selected from the set of portions of the 
original image for use as the current portion (step 196). A portion evaluation 
image is then presented (step 1 98) and the process of detecting whether a user 
5 input action is performed during presentation of the new current portion 
evaluation image (step 200) is performed. 

When controller 32 determines that the user has not made a user 
input action during a timeout period during which the of the current evaluation 
portion evaluation image is presented, controller 32 then proceeds to the step of 

10 determining an area importance image in the original image based upon the 

currently designated portion (step 206). Optionally, the area of importance can be 
used by controller 32, for example, for the purposes of cropping and/or resizing 
the original image (not shown). Also optionally, controller 32 can record 
information in association with the original image indicating the area of 

1 5 importance so that later editing processes can the area of importance to generate a 
cropped image (step also not shown). 

It will be appreciated that, using a method of the embodiment of 
Fig. 1 1, a display device 10 is provided that uses a one button control method for 
designating an area importance in an original image yet allows the user of the 

20 display device 10 to select from among portions of the original image that are not 
central to the original image. Such a one button embodiment has particular 
usefulness in display devices such as digital cameras, digital image and video 
players, and cellular telephones and provides a convenient and quick means for 
designating the area importance in an image without requiring precise alignment 

25 of borders of the area importance. Further, this embodiment does not require a 
user input action that is directional in that there is no inherent or apparent 
correlation between the direction of movement of the user input controls and the 
location of the designated portion. 

Fig. 1 2 shows another embodiment of a display device 1 0 of the 

30 invention. In the embodiment shown in Fig. 12, display device 10 provides a user 
input system 34 having two buttons that are used to make an area of importance 
designation: an advance button 210 and a save button 212. Fig. 13 shows one 
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embodiment of the method for determining an area of importance in original 
image using the user input system 34 shown in Fig. 12. As a shown in the 
embodiment of Fig. 13, and original images obtained (step 220) and an initial 
evaluation image is presented (step 222). 
5 Controller 32 monitors user input 34 during presentation of the 

initial evaluation image to detect whether the user display device 10 makes a user 
input action during presentation of the user input action (step 224). In this 
embodiment, controller 34 is adapted to detect two user input actions, an advance 
user input action and a save user input action during a timeout (e.g. ten seconds) at 

1 0 least part of which corresponds to a time during which the initial evaluation image 
is presented (step 222). The advance user input action is used by a user of display 
device 1 0 to request presentation of an evaluation image that is based upon a 
portion of the original image that is other than the currently presented portion. In 
the embodiment shown in Figs. 12 and 13, the advance user input action 

15 comprises depressing advance button 210. The save user input action is used to 
designate a currently presented portion. In the embodiment shown in Figs. 1 1 and 
12, the save user input action comprises actuation of the save button 212. It will 
be appreciated that where other forms of user input systems 34 are used, other 
inputs can be provided and used for the purpose of receiving advance user input 

20 action and a save user input action, some of which will be described in greater 
detail below. 

In the embodiment shown in Fig. 13, three options are available to 
a user of display device 1 0 during presentation of the evaluation image. A first 
option occurs when the user of display device 1 0 makes an advance user input 

25 action, controller 32 can proceed with the steps of presenting an portion 

evaluation images and possibly receiving a user selection (step 232-step 246) as 
will be as described in greater detail below. In a second option, the user can 
affirmatively designate the evaluation image that is based upon the entire original 
image in which case, the entire original image can be determined as an area 

30 importance (step 234). In a third option, the user can take no action within the 

timeout period (step 226) and, where this occurs, controller 32 can determine that 
no area of importance has been selected for the original image (step 228). 
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Alternatively, where the user fails to take any action within the timeout period, a 
controller 32 can determine that the entire original image is an area importance. 

When advance user input action is detected during the timeout 
period, controller 32 will cause an initial portion evaluation image to be presented 
5 on display 30 (step 234). The initial portion evaluation image can be a portion 
evaluation image that is based upon any of the predefined portions. In one 
embodiment, the initial portion evaluation image can comprise a portion 
evaluation image that is concentric with the original image, and, therefore, has the 
appearance of a conventionally zoomed image. However, this is not necessary. 

10 During presentation of the initial portion evaluation image, 

controller 32 monitors user input 34 to determine whether the user of display 
device 10 makes a user input action by actuating either the advance button 210 or 
the save button 212 (step 236). Where the advance button 210 is actuated during 
presentation of the initial portion evaluation image, controller 32 can designate a 

1 5 different one of the portions from the set of predetermined portions as a current 
portion of the original image (step 238) and will cause a portion evaluation image 
that is based upon the currently designated portion to be presented on display 30 
(step 240) so that the user of display device 10 can evaluate the currently 
designated portion. When save button 212 is actuated during presentation of the 

20 initial portion evaluation image (step 236), controller 32 will use the currently 
designated portion, in this case, the initial portion, to determine an area 
importance in the original image (step 248). Where no user input action is 
received, controller 32 can continue presentation of the current portion evaluation 
image for a timeout period. At the conclusion of the timeout period (step 246), 

25 controller 32 can optionally determine that no area of importance has been 

selected for the original image (step 228), can determine that the entire original 
image comprises an area of importance, or as shown, can determine that the 
portion of the image corresponding to the portion used to form the initial portion 
evaluation image is the area of importance, or can take such other action as may 

30 be dictated by a user preference or a manufacturer preference. 

During presentation of each portion evaluation image, controller 32 
monitors user input system 34 to determine whether the user of display device 10 
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actuates either of the advance button 210 (step 242) or the save button 212 (step 
244). Where the advance button 210 is actuated during presentation of the initial 
portion evaluation image, controller 32 can designate a different one of the 
portions from the set of predetermined portions as a current portion of the original 
5 image (step 238) and will cause a portion evaluation image to be presented on 

display 30 (step 240) so that the user of display device 10 can evaluate this portion 
evaluation image. Where the save button 212 is actuated during presentation of 
the initial portion evaluation image, controller 32 will use the currently designated 
portion, to determine an area of importance in the original image (step 248). At 

10 the conclusion of the timeout period (step 246), controller 32 can determine that 
the entire original image comprises an area of importance, controller 32 can 
determine that no area of importance has been selected for the original image (step 
228), or can take such other or as shown, controller 32 can determine that the 
portion of the image corresponding to the portion used to form the initial portion 

1 5 evaluation image is the area of importance (step 244), depending a user preference 
or a manufacturer preference. 

In one alternative embodiment of the method of Fig. 11, controller 
32 can be adapted to detect an advance user input and to begin presenting a 
sequence of portion evaluation images each portion evaluation image based upon 

20 one of a set of different portions of the original image. If the user makes a save 
input action during presentation of one of the portion evaluation images, the 
controller 32 will consider the user to have designated the currently presented 
portion of the original image for use in determining an area importance. If the 
user does not make a save input action during presentation of the portion 

25 evaluation images, controller 32 can determine that the entire image is an area of 
importance, or can determine that the user does not wish to make such a 
designation of an area importance. 

It will be appreciated that of the method shown in Fig. 13 can be 
performed using a variety of embodiments of display device 10 including any 

30 embodiment wherein the user interface input system 34 comprises at least two 

controls. For example, in a conventional camera embodiment of display device 10, 
the advance input can comprise the tele-photo zoom button 64 and the save button 
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can comprise a shutter trigger button 60. Similarly, in an embodiment where 
display device 10 comprises a cellular phone or other device having a keypad, any 
combination of two keys can be used for this purpose. 

In still another embodiment, the method of Fig. 1 1 can be practiced 
5 using a single user input control with a depression of the control acting as a 
advance user input and the release of the control acting as a save user input. In 
this way, an area can be designated using only a single touch of the user control. 

In yet another embodiment, shown in Figs. 14 and 15, display 
device 10 is shown comprising a personal digital assistant that incorporates a jog 

10 dial 300. Jog dial 300 is operable in at least one rotation direction (R) and one 
depression direction (D). In this embodiment jog dial 300 generates a rotation 
signal when jog dial 300 is rotated and a depression signal when jog dial 300 is 
moved in the depression direction. Controller 32 interprets the rotation signal as 
advance user input and controller 32 interprets the depression signal as a save user 

1 5 input. 

In the embodiments described above, the designated portion has 
been described in a manner that suggests that the shape of the designated portion 
is static with each portion defining and predefined area of the original image. This 
is not necessarily so. In the above described embodiments of the invention, a user 

20 of display device 1 0 can be permitted to designate the position and other 

characteristics of a designated portion. Figs. 14 and 15 illustrate one way in 
which this may be done. 

Fig. 14 shows a display device 10 presenting an evaluation image 
containing image information from the original image. As is shown in Fig. 14, a 

25 set of portions (260-276) are provided in the original image. Each portion (260- 
276) is located in the original image relative to a predefined anchor point (278- 
296). If jog dial 300 is rotated during presentation of the evaluation image, 
controller 32 will interpret this as an advance user input action and can determine 
that is necessary to present at least one portion evaluation image. The portion 

30 evaluation image can be associated with for example, portion 264. Accordingly, 
controller 32 causes the preparation and presentation of an initial evaluation image 
302 shown in Fig. 15. 
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In preparing initial evaluation image 302, controller 32 defines a 
portion shape 304 within the original image and locates the portion shape 304 
within the original image relative to anchor point 282. In this embodiment, 
controller 32 is further adapted to receive a shape designating input from user 
5 input system 34. The shape designating input can comprise, for example, 

magnification input that defines the size of the portion shape that is to be located 
relative to the anchor point. For example, as is shown in Fig. 1 5, after a user 
designates a portion 264 having anchor point 282, display device 10 presents the 
initial evaluation image 302. If during presentation of the evaluation image 302, 

10 jog dial 300 is depressed, controller 32 can determine that the user wishes to select 
designate the portion of the original image that corresponds to the portion shown 
in evaluation image 302 and can proceed as described above. If during 
presentation of the initial evaluation image 302, jog dial 300 is rotated, controller 
32 can define a new portion shape 306 or 308 relative to anchor point 290, and 

1 5 can cause the formation of a current evaluation image containing only image 

information that is within portion shape 306 or portion shape 308 as selected by 
the user when the user depresses jog dial 300. New portion shapes 306 or 308 are 
located with respect to anchor point 282 and other characteristics of the portions 
can be determined such as aspect ratio or overall form or pattern in like fashion. 

20 Accordingly, what is provided, is a display device 10 that can be 

used to designate portions of the original image as areas of importance without 
requiring directional inputs and positioning of elements. Controller 32 can be 
adapted to use this designation for variety of purposes including but not limited to 
generating metadata indicating the user's designation of an area importance can 

25 associating this metadata with the original image. In the case of a digital capture, 
the controller 32 can associate such area importance metadata with the image by 
storing the metadata in the captured digital image or by storing data that associates 
the digital image with a storage location containing the metadata area importance 
metadata so that the metadata can be retrieved later. Where the original image is 

30 captured for example on a photosensitive medium such as a photosensitive 

filmstrip, the metadata can be recorded on a photosensitive filmstrip optically 
recorded or magnetically recorded on a photographic conscript using techniques 
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known in the art. The area of importance data can also be used to crop the 
original image, to form thumbnail images for the original image, to adjust 
compression settings for storing the original image, or for variety of other uses. 

The invention has been described in detail with particular reference 
5 to certain preferred embodiments thereof, but it will be understood that variations 
and modifications can be effected within the spirit and scope of the invention. 



-33- 



PARTS LIST 





10 


disnlav device 




12 


digital camera 


5 


20 


body 




22 


image capture system 




23 


lens system 




24 


imaee sensor 




25 


lens driver 


10 


26 


Olgllcll UlUvvooUl 




27 


rangefinder 




28 


disnlav driver 




30 


disnlav 




32 


controller 


15 


34 


user input system 




36 






38 


viewfinder svstem 




40 


memorv 

iiiviiiv/i y 




48 


removable memory 


20 


50 


memory interface 




52 


remote memory system 




54 


communication module 




60 


trigger button 




62 


wide angle zoom button 


25 


64 


telephoto zoom button 




68 


edit button 



100 obtain original image step 

1 02 present evaluation image step 

1 04 detect user input action step 

30 1 06 designate portion step 

1 08 present portion evaluation image step 

1 1 0 detect a user reset action step 



-34- 





112 


determine area importance in original image 




114 


use determined area importance 




120 


original image 




122 


oortion 


5 


124 


oortion 




126 


portion 




128 


portion 




130 


portion 




132 


portion 


10 


134 


portion 




136 


portion 




138 


portion 




140 


portion 




142 


portion 


15 


144 


portion 




146 


portion 




148 


oortion 




1 50 






160 


portion evaluation image 


20 


162 


portion evaluation Image 




164 


portion evaluation image 




166 


mask 




168 


separator 




170 


area of the importance 


25 


180 


obtain original image step 




182 


present evaluation image 






present evaluation image step 




186 


detect user input action step 




188 


present initial portion evaluation image 


30 


190 


determine whether user input action has continued for at least an advance 
time period step 



-35- 
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